Discovering Multimodal Behavior in Ms. Pac-Man Through Evolution of Modular Neural Networks
نویسندگان
چکیده
Ms. Pac-Man is a challenging video game in which multiple modes of behavior are required: Ms. Pac-Man must escape ghosts when they are threats and catch them when they are edible, in addition to eating all pills in each level. Past approaches to learning behavior in Ms. Pac-Man have treated the game as a single task to be learned using monolithic policy representations. In contrast, this paper uses a framework called Modular Multi-objective NEAT (MM-NEAT) to evolve modular neural networks. Each module defines a separate behavior. The modules are used at different times according to a policy that can be human-designed (i.e. Multitask) or discovered automatically by evolution. The appropriate number of modules can be fixed or discovered using a genetic operator called Module Mutation. Several versions of Module Mutation are evaluated in this paper. Both fixed modular networks and Module Mutation networks outperform monolithic networks and Multitask networks. Interestingly, the best networks dedicate modules to critical behaviors (such as escaping when surrounded after luring ghosts near a power pill) that do not follow the customary division of the game into chasing edible and escaping threat ghosts. The results demonstrate that MM-NEAT can discover interesting and effective behavior for agents in challenging games.
منابع مشابه
Evolution Strategies for Evolving Artificial Neural Networks in an Arcade Game
The aim of this paper is to use a simple but powerful evolutionary algorithm called Evolution Strategies (ES) to evolve the connection weights and biases of feed-forward artificial neural networks (ANN) and to examine its learning ability through computational experiments in a non-deterministic and dynamic environment, which is the well-known arcade game called Ms. Pac-man. The resulting algori...
متن کاملEvolving Multimodal Behavior Through Subtask and Switch Neural Networks
While neuroevolution has been used successfully to discover effective control policies for intelligent agents, it has been difficult to evolve behavior that is multimodal, i.e. consists of distinctly different behaviors in different situations. This article proposes a new method, Modular NeuroEvolution of Augmenting Topologies (ModNEAT), to meet this challenge. ModNEAT decomposes complex tasks ...
متن کاملمعرفی شبکه های عصبی پیمانه ای عمیق با ساختار فضایی-زمانی دوگانه جهت بهبود بازشناسی گفتار پیوسته فارسی
In this article, growable deep modular neural networks for continuous speech recognition are introduced. These networks can be grown to implement the spatio-temporal information of the frame sequences at their input layer as well as their labels at the output layer at the same time. The trained neural network with such double spatio-temporal association structure can learn the phonetic sequence...
متن کاملSingle- versus Multiobjective Optimization for Evolution of Neural Controllers in Ms. Pac-Man
متن کامل
Evolving a Neural Network Location Evaluator to Play Ms. Pac-Man
Ms. Pac-Man is a challenging, classic arcade game with a certain cult status. This paper reports attempts to evolve a Pac-Man player, where the control algorithm uses a neural network to evaluate the possible next moves. The evolved neural network takes a handcrafted feature vector based on a candidate maze location as input, and produces a score for that location as output. Results are reporte...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- IEEE transactions on computational intelligence and AI in games
دوره 8 1 شماره
صفحات -
تاریخ انتشار 2016